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Objective: Verbal autopsy (VA) is a systematic approach for determining causes of death (CoD) in populations 
without routine medical certification. It has mainly been used in research contexts and involved relatively 
lengthy interviews. Our objective here is to describe the process used to shorten, simplify, and standardise the 
VA process to make it feasible for application on a larger scale such as in routine civil registration and vital 
statistics (CRVS) systems. 

Methods: A literature review of existing VA instruments was undertaken. The World Health Organization 
(WHO) then facilitated an international consultation process to review experiences with existing VA 
instruments, including those from WHO, the Demographic Evaluation of Populations and their Health in 
Developing Countries (INDEPTH) Network, InterVA, and the Population Health Metrics Research 
Consortium (PHMRC). In an expert meeting, consideration was given to formulating a workable VA CoD 
list [with mapping to the International Classification of Diseases and Related Health Problems, Tenth Revision 
(ICD-10) CoD] and to the viability and utility of existing VA interview questions, with a view to undertaking 
systematic simplification. 

Findings: A revised VA CoD list was compiled enabling mapping of all ICD-10 CoD onto 62 VA cause 
categories, chosen on the grounds of public health significance as well as potential for ascertainment from VA. 
A set of 221 indicators for inclusion in the revised VA instrument was developed on the basis of accumulated 
experience, with appropriate skip patterns for various population sub-groups. The duration of a VA interview 
was reduced by about 40% with this new approach. 

Conclusions: The revised VA instrument resulting from this consultation process is presented here as a means 
of making it available for widespread use and evaluation. It is envisaged that this will be used in conjunction 
with automated models for assigning CoD from VA data, rather than involving physicians. 
Keywords: verbal autopsy; cause of death; vital registration; civil registration; vital statistics; World Health Organization; Inter VA 
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Information on causes of death (CoD) is essential for 
planning, implementing, monitoring, and evaluating 
public health at all levels. However, death registration 
and CoD determination do not happen for many deaths 
occurring in low- and middle-income countries (LMICs), 
and the deaths of poorer people are much less likely to be 
recorded, compounding inequalities. Statistical modelling 
is used to fill the data gaps, for example, for maternal 
deaths and malaria mortality. Facilitating complete and 
accurate CoD determination and death registration in 
LMICs is therefore a high priority. In the medium-term, 
this will involve applying verbal autopsy (VA) not only 
in surveillance sites and household surveys but also as 
a routine part of civil registration and vital statistics 
(CRVS) systems (1, 2). 

VA ascertains probable CoD through interviews car- 
ried out with caretakers of the deceased or witnesses 
of deaths. The method uses questionnaires to eUcit 
pertinent information on signs, symptoms, and circum- 
stances leading to death, generically described as indica- 
tors, which are subsequently interpreted into CoD. VA 
has been increasingly used in various contexts including 
disease surveillance, sample registration systems, out- 
break investigation, and measuring the impact of public 
health interventions. Because vital registration coverage 
has not significantly improved in most LMICs, VA data 
collection has been conducted in a variety of settings 
such as clinical trials and large-scale epidemiological 



studies; demographic surveillance systems; national sam- 
ple surveillance systems; and household surveys. The 
expanding use of VA in generating mortaUty data has led 
to a proliferation of different VA instruments (comprising 
a set of questions/indicators that elicit pertinent infor- 
mation on signs, symptoms and circumstances preceding 
death and a corresponding list of CoD) that has impaired 
data comparability across sites and over time. Limited 
attention has been given to standardization of CoD 
interpretation from VAs (3). 

Users have different perspectives on the required level 
of accuracy and categories of cause-specific mortality 
data, with corresponding impacts on desirable charac- 
teristics of VA instruments (4). However, the need for 
regular nationally representative cause-specific mortal- 
ity data in settings where a significant proportion of 
deaths are not medically certified can only be met by 
death registration including VA as part of national 
CRVS systems. This requires simpler VA instruments 
and operating procedures that can produce timely, readily 
usable and reliable cause-specific mortality data. 

To produce a simplified VA instrument, the World 
Health Organization (WHO) carried out a systematic re- 
view of VA instruments and procedures, followed by an ex- 
pert consultation. Based on accumulated experience from 
widely-used and validated VA procedures, consensus was 
reached on a simplified VA instrument for routine use in 
CRVS systems where deaths are not medically certified. 
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Revision of WHO verbal autopsy instrument 



The 2012 WHO VA instrument comprises a short CoD 
Ust aUgned to the International Classification of Diseases 
and Related Health Problems, Tenth Revision (ICD-10) 
that is ascertainable fi-om a limited number of indica- 
tors and amenable to automated processing. The design 
allows adding a narrative and locally relevant questions 
and diagnoses as needed. The rationale and processes 
used to develop the 2012 WHO VA instrument are pre- 
sented in this article. 

VA instruments and procedures 

The WHO first encouraged the use of lay reporting of 
health information in 1956, and from then through the 
1990s, developed lay reporting forms and published 
key design features for studies based on VA methods. 
With the expanding diversity and use of VA instruments, 
demands for standardization led to the development of 
the WHO VA standards in 2007 that included (5): 

(1) VA questionnaires for three age groups (under 4 
weeks; 4 weeks to 14 years; and 15 years and above); 

(2) CoD certification and coding resources consistent 
with ICD-10; and 

(3) A CoD Hst for VA prepared according to the 
ICD-10. 

The 2007 WHO VA standards were partially based on 
a VA instrument developed by the London School of 
Hygiene and Tropical Medicine (LSHTM). The WHO 
standards expected that up to three physicians trained in 
VA coding would independently review individual ques- 
tionnaire data - known as physician-certified VA (PCVA). 
This procedure has been used by the International Net- 
work for the Demographic Evaluation of Populations 
and their Health in Developing Countries (INDEPTH)' 
and by the Sample Vital Registration with Verbal 
Autopsy (SAVVY).^ 

However, since PCVA is time-consuming and expensive, 
computerized coding of VA (CCVA) methods for inter- 
preting VA data have been investigated. Validated CCVA 



'iNDEPTH (www.indepth-network.org) is a network of member 
centres that conduct longitudinal health and demographic 
evaluation of populations in LMICs. INDEPTH has built a 
network of 44 health and demographic surveillance systems 
(HDSS) across 20 countries in Africa, Asia, and Oceania. The 
network strengthens capacity of HDSS centres, and mounts 
multicentre research to guide health priorities and policies in 
LMICs, based on up-to-date empirical scientific evidence. The 
network uses VA as a method for determining CoD. 
^SAVVY, proposed by MEASURE Evaluation and the International 
Programs Center, U.S. Census Bureau, is a system to generate 
reliable information on mortality levels and CoD at the national 
level. The SAVVY resource library is a series of best practice 
manuals and methods for improving the quality of vital statistics 
where high coverage of civil registration and good CoD data are not 
available. A SAVVY system collects mortality data from a number of 
sites throughout a country using multistage probabihty sampling. 
SAVVY Methods include determination of CoD with VA. 



methods can be algorithmic or probabilistic. Algorithmic 
methods follow a set of predefined diagnostic criteria 
that can be expert- or data-derived. The Tariff method is 
an additive algorithm that uses Tariff scores reflecting 
the importance and uniqueness of each symptom to each 
CoD. The Artificial Neural Network (ANN) method 
uses computer algorithms (machine learning), applying 
non-linear statistics to pattern recognition. The Random 
Forests method is a machine learning method for inter- 
preting VA based on patterns of indicators from a 
'training dataset' (6). Whereas algorithmic methods result 
in binary outcomes (yes or no) for a single CoD, pro- 
babilistic methods determine the probability of a range 
of multiple causes. The InterVA method applies Bayesian 
probabilistic methods to a matrix of indicators and CoD, 
using conditional probabilities derived from available 
data and expert opinion. This method has been available 
in the public domain since 2006 (7, 8). King and Lu's 
algorithmic method is able to estimate cause-specific 
mortality fractions (CSMFs) without individual CoD 
assignment. The Simplified Symptom Pattern (SSP) 
method is a data-driven Bayesian approach that combines 
the King and Lu and InterVA methods. 

Review of utilization of VA instruments and 
procedures 

Despite attempts to standardize and harmonize VA 
instruments, there are multiple instrimients in use (9-1 1). 
We conducted a systematic literature review to deter- 
mine how VA instruments have been used and the uptake 
of the WHO VA standards pubHshed in 2007. 

The review included studies reported in peer-reviewed 
journals from 1986 up to early 2012. Figure 1 illustrates 
the review process. The WHO instruments and the 
three related ones briefly described above (INDEPTH, 
SAVVY and LSHTM) were included in the review. 
Instruments described as adapted from these were also 
included. Studies that did not provide details of the 
instrument used were excluded. A brief description of the 
125 eligible studies is available as a Supplementary File. 

Some studies applied different VA interpretation meth- 
ods on the same dataset and were counted as a single 
study for the review of the use of the VA instruments. 
The selected VA instruments or their adaptations were 
reported to be used by 1 12 studies in 41 countries. Table 1 
summarizes the identified studies, data collection period, 
and number of deaths certified, by each VA instrument. 
VA was mostly used as a research tool in longitudinal 
health and demographic surveillance and in interven- 
tion or epidemiological studies. The first study identified 
used an adapted version of an early WHO instrument to 
certify perinatal deaths in Nepal in 1989 (12). From the 
112 reviewed studies, 104 reported the number of deaths 
certified, totalling 159,316. Studies using the INDEPTH 
instrument certified the largest number of deaths, ranging 
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Search Terms 



Verbal autopsy; post mortem interview; cause of death; mortality surveillance 



Literature Databases searched and number of citations found 


1) PubMed 


2) Popline 


3) LILACS 


1) 465 


2) 203 


3)9 



Titles and abstracts of all 677 publications were read by one of the authors, and all articles 
on studies reporting to have used the WHO, INDEPTH, SAVVY, or LSHTM VA instruments or 
adaptations of the instruments were considered eligible for review. A total of 340 articles 
were excluded from the review based on the following criteria: (i) no description of VA 
instrumentused; (ii) did not use the WHO, INDEPTH, SAVVY, or LSHTM VA instruments or 
adapted versions of these instruments; (ill) not published in English, French, Spanish 
or Portuguese. 





\/ 




The retained 125 studies were reviewed by one of the authors. Additional publications known 
to any of the authors but not identified through the literature search, and references quoted 
in original publications, were also reviewed. 



Fig. I. Illustration of literature search and review process. 



from 100 to 38,306 deaths with a mean of 4269.4, 
totalling 72,579 deaths (Table 1). 

VA has also been applied in national health surveys. 
In most surveys (e.g. Nepal Demographic and Health 
Survey (DHS) 2006, Ghana DHS 2008, Bangladesh DHS 
2011), this involves the identification of deaths among 



children under 5 years in either the household schedule 
or the individual interview of women of reproductive age, 
followed by administration of a VA module. In Uganda, 
deaths among children under 5 identified in the DHS 
in 2007 were followed up in a subsequent survey in 2008. 
In the Afghanistan Mortality Survey 2010, a VA was 



Table 1. Summary characteristics of reviewed studies (w = 1 12) by type of VA instrument' 



% of studies with sites 

Number of Data collection Mean and range of number Central and 

identified studies period of deaths certified Africa Asia South America 



WHO VA instrument 


31 (27.7%) 


1992-2010 


620.1 (23^ 239) 


61.3 


32.3 


16.1 


Adapted WHO VA instrument 


42 (37.5%) 


1989-2010 


1 347.5 (2-12 542) 


35.7 


59.5 


7.1 


INDEPTH VA instrument 


17 (15.2%) 


1996-2009 


4269.4 (100-38 306) 


64.7 


35.3 


0 


Adapted INDEPTH VA instrument 


9 (8%) 


1999-2010 


590.7 (164-1 816) 


100 


0 


0 


SAVVY instrument 


1 (0.9%) 


2007-2010 


145 


100 


0 


0 


Adapted SAVVY instrument 


3 (2.7%) 


2006-2008 


258 (252-264) 


33.3 


0 


66.7 


LSHTM VA instrument 


5 (4.5%) 


1992-2002 


407.3 (40-796) 


80 


20 


0 


Adapted LSHTM VA instrument 


4 (3.6%) 


2003-2007 


2 304.3 (1 084-5 160) 


25 


75 


0 



^Percentages of studies conducted amount to more than 100% because some multicentre studies had sites in more than one continent. 
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Reviewed verbal autopsy studies by site, size and instrument 1989-2010 




Number of autopsies 

o <= 100 



O 

o 
o 

o 



101 - 500 
501 - 1000 
1001 - 2500 
2501 - 5000 
> 5000 




Tool type used 

INDEPTH VA tool 
INDEPTHVA tool, adapted 
WHO VA tool 
WHO VA tool, adapted 

WHO VA tool and INDEPTH VA tool combined 
LSHTMVAtool 
LSHTM VAtool, adapted 
SAVVY VA tool, adapted 
SAVVY VA tool 

Not applicable 



875 1,750 3,50« il 



The boundaries and names shown and the designations used on this map do not imply the expression of any opinion whatsoever Data Source: World Health Organization 

on the part of the World Health Organization conoeming Itie legal status of any country, territory, city or area or of its authorities. Map Production: Public Health Infomralion 

or concerning the delimitation of its frontiers or boundaries. Dotted and dashed lines on maps represent approximate border iines and Geographic Information Systems (GIS) 

for which there may not yet be fuil agreement World Health Organization 
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Fig. 2. Global distribution of verbal autopsy studies. 

administered for deaths of all ages. In Mozambique, a 
post census VA was conducted in 2008. All surveys ask 
for medical certification of the CoD, but the majority rely 
on VA using a variety of questionnaires. 



Table 1 and Fig. 2 show that the majority of reviewed 
studies had sites in Africa (54.5%) and Asia (40.2%), while 
some were conducted in Central and South America 
(8.9%). The majority of studies using the WHO (61.3%), 



< 
> 



14 
12 

10 
8 
6 



J_L 




I 



I 



1 



1989 1991 1993 



1995 



1997 



1999 2001 
Years 



2003 



2005 2007 2009 



I WHO VA instrument 
I INDEPTH VA instrument 
I SAVVY instrument 
I LSHTM VA instrument 



Adapted WHO VA instrument 
Adapted INDEPTH VA instrument 
Adapted SAVVY instrument 
Adapted LSHTM VA instrument 



Fig. 3. Use of different VA instruments over time. 
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INDEPTH (64.7%) or adapted versions of the INDEPTH 
instruments (100%) were in Africa; studies using tlie 
WHO (32.3%) or INDEPTH (35.3%) instruments also 
liad sites in Asia. 

Use over time for eacli VA instrument is sliown in 
Fig. 3. Publications using the WHO and INDEPTH 
VA instruments (and adaptations) increased around 
1999, peaking between 2003 and 2005. There have been 
a limited number of published studies using other 
instruments. Since the publication of the WHO VA 
standards in 2007, 17 studies have been conducted which 
used the WHO VA instrument and adaptations (12/17); 
the INDEPTH instrument and adaptations (2/17); and 
the SAVVY instrument and adaptations (3/17). While 
these figures show that the majority of studies since 2007 
have used the WHO VA instrument and adaptations, 
it is difficult to assess the level of uptake of the WHO VA 
standards, as trends in more recent data collection years 
may be difficult to interpret due to delays in pubhcation 
of results, particularly given delays in PCVA interpreta- 
tion in some sites. 

Age groups were reported by 110 studies. For compar- 
isons, age groups were categorized non-exclusively as: 
stillbirths; under 4 weeks; 4 weeks to 5 years; under 15 
years; 15 years and above; maternal deaths; and all age 
groups (Table 2). VA instruments have mostly been used 
for 15 years and above (26.4%) and for all age groups 
(22.7%). Deaths in children under 5 years old (18.2%) and 
neonates (18.2%) have also been widely studied. 

The most common interpretation method (more than 
one was used in some studies) was the PCVA (82.9%), 
followed by probabiKstic methods (11.7%), and algo- 
rithms (10.8%) (2). Of probabilistic methods, InterVA 



was most used (61.5%). Only one study used ANN, 
Random Forest, SSP, Tariff, or King and Lu methods 
to ascertain CoD. 

Validity studies for VA procedures are fraught with 
difficulties since there is no widely available gold standard, 
particularly for the majority of LMICs deaths not oc- 
curring in health facilities (13). The validity of VA is 
typically assessed by comparing hospital medical records 
as gold standard diagnoses for CoD, as well as by making 
between-method comparisons (e.g. between PCVA and 
CCVA). The validity of the overall VA process is influenced 
by the design and content of the questionnaires, field pro- 
cedures, data interpretation methods, actual CoD pat- 
terns, and characteristics of the deceased (14). 

Of the 125 studies reviewed, 26 assessed performance 
of VA procedures in certifying CoD (studies using the 
same VA dataset but different CoD interpretation meth- 
ods and/or assessing different vahdation parameters were 
included in the review and counted as individual studies) 
(Tables 3 and 4). Apart from adapted versions of the 
LSHTM VA instrument, all other types of VA instruments 
have been validated at least once by these studies. The 
majority of studies vahdating VA procedures have used 
the WHO VA instrument (6/26) and their adapted versions 
(10/26). A similar review by Chandramohan et al. in 1994 
identified almost no information on the validity of VA for 
adult deaths (7). Our review identified that this trend has 
shifted with most of the 26 studies having assessed the 
performance for all CoD (21/26), in adults (10/26) and 
in all age groups (10/26). These studies used a variety of 
measures, including: sensitivity (14/26); specificity (14/26); 
positive predictive value (PPV) (8/26); negative predictive 
value (NPV) (4/26); cause-specific fractions (CSF) (1/26); 



Table 2. Age groups studied by type of VA instrument (n = 110)^ 







Under 


Under 


Under 


Aged 1 5 years 


Maternal 


All age 




Stillbirths 


4 weeks 


5 years 


1 5 years 


and above 


deaths 


groups 


WHO VA instrument 


13.3% (4/30) 


20.0% (6/30) 


30.0% (9/30) 


3.3% (1/30) 


13.3% (7/30) 


10.0% (3/30) 


10.0% (3/30) 


Adapted WHO VA 


7.3% (3/41) 


24.4% (10/41) 


17.1% (7/41) 


0% (0/41) 


17.1% (7/41) 


17.1% (7/41) 


26.8% (11/41) 


instrument 
















INDEPTH VA 


5.9% (1/17) 


11.8% (2/17) 


11.8% (2/17) 


11.8% (2/17) 


17.6% (3/17) 


0% (0/17) 


47.1% (8/17) 


instrument 
















Adapted INDEPTH 


0% (0/9) 


0% (0/9) 


22.2% (2/9) 


33.3% (3/9) 


33.3% (3/9) 


0% (0/9) 


11.1% (1/9) 


VA instrument 
















SAVVY instrument 


0% (0/1) 


0% (0/1) 


0% (0/1) 


0% (0/1) 


100.0% (1/1) 


0% (0/0) 


0% (0/0) 


Adapted SAVVY 


33.3% (1/3) 


66.7% (2/3) 


0% (0/3) 


0% (0/3) 


33.3% (1/3) 


0% (0/3) 


0% (0/3) 


instrument 
















LSHTM VA instrument 


0% (0/5) 


0% (0/5) 


0% (0/5) 


0% (0/5) 


100.0% (5/5) 


0% (0/5) 


0% (0/5) 


Adapted LSHTM 


0% (0/4) 


0% (0/4) 


0% (0/4) 


0% (0/4) 


50.0% (2/4) 


0% (0/4) 


50.0% (2/4) 


VA instrument 
















Total 


8.2% (9/110) 


18.2% (20/110) 


18.2% (20/110) 


5.5% (6/110) 


26.4% (29/110) 


9.1% (10/110) 


22.7% (25/110) 



^Percentages do not add up to 100% as some studies determined CoD in more than one age group. 
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Table 3. Main characteristics of reviewed VA validation studies (n =26) 



Measures of validity 















Concordance 






Cause-specific 




Cause-specific concordance 


Validated 














between VA OoMr 






and average 




correlation coefficients of 


against 


Number of validation 












ana uoivir irom 




Kappa 


chance-corrected 


r^Q^^p 


estimated CSMFs compared 


hospital 


studies 


Sensitivity 


Specificity 


PPV 


NPV 


CSF 


validation data 


curve 


statistics 


concordance 


accuracy 


to true CSMFs 


CoD data 


WHO VA instrument {n =6) 


6/6 


6/6 


2/6 


2/6 


1/6 


1/6 


1/6 


0/6 


0/6 


0/6 


0/6 


6/6 


Adapted WHO VA 


3/10 


2/10 


2/10 


1/10 


0/10 


2/10 


0/10 


2/10 


5/10 


6/10 


6/10 


9/10 


instrument (n = 1 0) 


























INDEPTH VA instrument 


0/3 


0/3 


0/3 


0/3 


0/3 


2/3 


0/3 


2/3 


0/3 


0/3 


0/3 


0/3 


(n=3) 


























Adapted INDEPTH VA 


1/1 


1/1 


1/1 


0/1 


0/1 


1/1 


1/1 


1/1 


0/1 


0/1 


0/1 


0/1 


instrument (n =1) 


























SAVVY instrument (n =1) 


1/1 


1/1 


1/1 


1/1 


0/1 


1/1 


1/1 


1/1 


0/1 


0/1 


0/1 


1/1 


Adapted SAVVY instrument 


0/1 


1/1 


1/1 


0/1 


0/1 


0/1 


0/1 


0/1 


0/1 


0/1 


0/1 


0/1 


(1-1) 
LSHTM VA Instrument 


3/4 


3/4 


1/4 


0/4 


0/4 


3/4 


0/4 


1/4 


0/4 


0/4 


0/4 


4/4 


(n=4) 


























Total (n=26) 


14/26 


14/26 


8/26 


4/26 


1/26 


10/26 


3/26 


7/26 


5/26 


6/26 


6/26 


20/26 



roo 



Table 4. List of reviewed VA validation studies (n =26) 



Instrument and source 



VA interpretation 
method 



Number of 
deaths certified 



CoD 
studied 



Age groups 
studied 



Validity and reliability parameters 



WHO VA instrument (1 5) Physician review 

WHO VA instrument (1 6) Algorithms 

WHO VA instrument (17) Physician review 

WHO VA instrument (1 8) Physician review 

WHO VA instrument (1 9) Physician review 

WHO VA instrument (20) Physician review 



Adapted WHO VA 
instrument (21) 

Adapted WHO VA 
instrument (22) 

Adapted WHO VA 
instrument (23) 

Adapted WHO VA 

instrument (24) 
Adapted WHO VA 

instrument (8) 
Adapted WHO VA 

instrument (25) 
Adapted WHO VA 

instrument (26) 
Adapted WHO VA 

instrument (27) 
Adapted WHO VA 

instrument (28) 

Adapted WHO VA 
instrument (6) 

IN DEPTH VA 
instrument (29) 



Physician review 

Physician review 

Physician review 
and InterVA 

Physician review 

InterVA, physician 
review and SP method 
King Lu method and 
physician review 
Physician review 

Tariff method and 
physician review 
SSP method and 
physician review 

Random Forests 
method and physician 
review 

Physician review and 
InterVA 



225 
1115 

719 

763 
1251 
36 

255 

219 

734 

9817 
12542 
12 542 
12 542 
12 542 
12 542 

12 542 

1 823 



All causes 
Diarrhoea and 
pneumonia 
All causes 

Stroke 
All causes 
Selected childhood 
illnesses 
All causes 

All causes 

All causes 

All causes 
All causes 
All causes 
All causes 
All causes 
All causes 

All causes 

All causes 



Sensitivity, Specificity, PPV, NPV, ROC curves 
Sensitivity, Specificity, CSMF 



Stillbirths 
Children under 
5 years 

Children under Sensitivity, Specificity, PPV, Difference between CSMF estimated by VA 

5 years and true CSMF in validation data 

Adults Sensitivity, Specificity 

Stillbirths and neonates Sensitivity, Specificity 

Children under 12 Sensitivity, Specificity, PPV, NPV 

years old 



All age groups 
Adults 

Stillbirths and 
neonates 

All age groups 

All age groups 

All age groups 

All age groups 

All age groups 

All age groups 

All age groups 



Children under 
5 years and adults 



Sensitivity, Specificity 

Sensitivity, Specificity, PPV, NPV, Kappa statistics 

Concordance of CSMFs estimated by InterVA and physician review, 
Level of agreement between InterVA and physician assigned CoD using 
Kappa statistics 

Sensitivity, PPV, Concordance of CSMFs estimated by physician review 
and medical record diagnoses 

Average of cause-specific chance-corrected concordance, CSMF 
accuracy, relationship between estimated CSMFs and true CSMFs 
CSMFs accuracy, relationship between estimated CSMFs and true 
CSMFs 

Average of cause-specific chance-corrected concordance, CSMFs 
accuracy. Relationship between estimated CSMFs and true CSMFs 
Average of cause-specific chance-corrected concordance, CSMFs 
accuracy. Relationship between estimated CSMFs and true CSMFs 
Average of cause-specific chance-corrected concordance, CSMFs 
accuracy. Relationship between estimated CSMFs and true 
CSMFs 

Average of cause-specific chance-corrected concordance, CSMFs 
accuracy. Relationship between estimated CSMFs and true 
CSMFs 

Level of agreement between InterVA and physician assigned CoD using 
Kappa statistics. Concordance of CSMFs estimated by InterVA and 
physician review 



Table 4 (Continued) 



Instrument and source 



VA interpretation Number of CoD 
metliod deaths certified studied 



Age groups 
studied 



Validity and reliability parameters 



INDEPTH VA instrument Physician review and 

(30) InterVA 

INDEPTH VA instrument Physician review and 

(31) InterVA 
Adapted INDEPTH VA InterVA 

instrument (32) 

SAVVY instrument (33) Physician review and 
InterVA 



Adapted SAVVY 

instrument (34) 
LSHTM VA instrument 

(35) 

LSHTM VA instrument 
(36) 

LSHTM VA instrument 
(37) 



LSHTM VA instrument 
(38) 



Physician review 

Physician review and 
algorithms 

Physician review and 
expert algorithms 
Physician review, 
expert algorithms 
and data-derived 
algorithms 
Data-derived 
algorithms 



10 267 All causes 
289 All causes 

193 HIV/AIDS 



145 

264 
615 
796 
796 

40 



All causes 

HIV/AIDS 
All causes 
All causes 
All causes 

All causes 



All age groups 
All age groups 
Adults 

Adults 

Adults 
Adults 
Adults 
Adults 

Adults 



Level of agreement between InterVA and physician assigned CoD using 
Kappa statistics 

Concordance of CSMFs between InterVA and physician review 

Sensitivity, Specificity, PPV, Concordance of CSMFs between InterVA 
and the reference standard. Level of agreement between InterVA and 
reference standard CoD using Kappa statistics, ROC curves 
Sensitivity, Specificity, PPV, NPV, ROC curves, Level of agreement 
between InterVA, physician review and hospital CoD using Kappa 
statistics. Concordance of CSMFs between InterVA, physician review 
and hospital CoD 
Specificity, PPV 

Sensitivity, Specificity, Concordance of CSMF obtained using the 
data-derived algorithms with the CSMF obtained using physician review 
Sensitivity, Specificity, PPV, Concordance of CSMFs between physician 
review, algorithms and hospital CoD 

Sensitivity, Specificity, Concordance of CSMFs between physician 
review, algorithms and hospital CoD 



Kappa statistics 



Jordana Leitao et al. 



concordance between CSMF estimated by VA and CSMF 
from the validation data (10/26); areas under the receiver 
operator characteristic (ROC) curve (3/26); kappa statis- 
tics (7/26); cause-specific and average chance-corrected 
concordance (5/26); CSMF accuracy (6/26); and cause- 
specific concordance correlation coefiicients of estimated 
CSMFs compared to true CSMFs (6/26). The ability of 
these studies to adequately validate VA has often been 
limited by small sample sizes, affecting reliability of meas- 
ures such as sensitivity and specificity, and the absence 
of certain causes from hospital data. Most studies (20/26) 
relied on hospital CoD data as the standard measure 
of validity. Exceptions included comparison studies using 
the InterVA method (7/26), where in five studies the 
reliability of VA procedure was assessed by the concor- 
dance of CSMFs estimated by InterVA and PCVA. The 
review found two studies vahdating the performance of 
InterVA against hospital CoD data (8, 33). 

Our review of VA studies pubhshed up to 2012 highhghts 
variability in the selection, development, and use of VA 
instruments, as well as in methods of assessment. The 
review established that there are many adaptations of 
standard VA instruments. Although instruments may 
need to be adapted to local contexts, the extent of 
modifications was not reported by studies and their impact 
on VA performance and accuracy are not known. The 
review was hindered by an absence of information on the 
VA instrument used by a substantial number of studies. 
The lack of systematic detailed information on methods 
used undermines the value of experience sharing on use 
of VA instruments and limits a more accurate under- 
standing of the use of the different instruments and uptake 
of VA guidelines. Some reports on using VA may have been 
missed if written in other languages or as yet unpublished. 

Simplification of VA standards: tlie 2012 WHO 
VA instrument 

In December 2011, following the above review process, 
consensus over a simplified VA instrument was reached 
among 37 experts from 15 countries in a meeting organ- 
ized by WHO in collaboration with the University of 
Queensland, the Health Metrics Network and INDEPTH. 

Table 5. Pattern of indicators by age group 



The meeting was followed by a 2-day workshop during 
which the outcomes of the discussions were consolidated. 
Participants included key stakeholders, researchers, and 
those who work routinely with VA instruments. The 2012 
WHO VA instrument comprises a total of 22 1 CoD-related 
indicators to certify 62 CoD. The instrument is designed 
primarily for electronic data capture, and WHO data 
collection software will facilitate this on generic mobile 
devices. CoD interpretation software also allows assess- 
ment without physicians, reducing cost and time lag in 
VA interpretation, and enhancing comparability across 
different settings and over time. For those wanting to 
use paper capture and PCVA, simplified sample ques- 
tionnaires have been developed for three age groups: 
under 4 weeks; 4 weeks to 14 years; and 15 years and 
over, which are available with all other aspects of the 
2012 WHO VA instrument at www.who.int/healthinfo/ 
statistics/verbalautopsystandards 

As determined by extensive skip patterns, the maximum 
number of questions to be asked for any death ranges 
from 104 for a neonatal death to 130 for a maternal death 
(Table 5). Although users may need to add locally relevant 
questions, the instrument as defined here should be 
regarded as the core. 

Simplified list of CoD 

To develop a VA instrument appropriate for strengthen- 
ing countries' CRVS systems, we simplified the WHO 
VA standards; this commenced with generating a shorter 
list of CoD. Three main criteria characterized essential 
CoD: 

(1) Importance: most frequent CoD of global public 
health relevance (e.g. acute respiratory infections); 

(2) Diagnostic Feasibility: CoD associated with recog- 
nizable symptoms ascertainable by VA (e.g. HIV/ 
AIDS); and 

(3) Potential for intervention: CoD can be addressed by 
public health interventions (e.g. diarrhoeal diseases). 

Comparison of the results of most widely used and 
validated VA instruments and interpretation approaches 



Number of indicators 



Age group CoD-related 



Sl<ip level First Second Third Fourth Total Personal Respondent Context Total 



15+ years 


56 


37 


27 


10 


130 


26 


3 


10 


169 


4 weeks-1 4 years 


34 


35 


22 


10 


101 


26 


3 


10 


140 


Under 28 days 


44 


35 


15 


10 


104 


26 


3 


10 


143 


Total 


93 


87 


31 


10 


221 
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Table 6. Correspondence of CoD between the 2007 WHO VA standards, InterVA and PHMRC VA instruments, the 2004 GBD, 
and their reported percentage in 125 reviewed VA studies 



2007 WHO VA standards 2004 GBD InterVA PHIVIRC VA instrument % Reported in various studies 



Infectious and parasitic diseases 

Sepsis • ■ 10.4 

Acute respiratory infection, including pneumonia * * " 37.6 

HIV/AIDS related death • • • 36.8 

Intestinal infectious diseases * * " 40.8 

Malaria • * " 33.6 

Measles • • " 10.4 

Meningitis • * " 30.4 

Tetanus • 4.8 

Pulmonary tuberculosis • * " 35.2 

Typhoid and Paratyphoid 0.8 

Pertussis • 2.4 
Leishmaniasis 0 
Viral hepatitis 6.4 
Arthropod-borne viral fevers and viral haemorrhagic fevers " 4.0 

Other infectious disease, unspecified * " 21.6 

Neoplasms 

Oral neoplasms " 4.0 

Digestive neoplasms • " 12.0 

Malignant neoplasm of rectum and anus " " 4.8 

Respiratory neoplasms ■ " 8.0 

Breast neoplasms " " 4.8 

Reproductive neoplasms • " 1 0.4 

Melanoma of skin 0 
Neoplasm of lymphoid, haematopoietic and related tissue 0.8 
Other and unspecified neoplasms • " 20.0 

Nutritional and endocrine disorders 

Severe anaemia 9.6 

Severe malnutrition • • 16.0 

Diabetes mellitus • • " 14.4 

Other and unspecified nutritional and endocrine disorders 1.6 

Diseases of circulatory system 

Acute cardiac disease * * " 16.0 

Sickle cell • 0.8 

Cerebrovascular disease * * " 22.4 

Other and unspecified cardiac disease " 44.0 

Respiratory disorders 

Chronic obstructive pulmonary disease (COPD) " 6.4 

Asthma • 4.8 

Other and unspecified respiratory disease * 20.8 

Gastrointestinal diseases 

Acute abdominal condition 6.4 

Chronic liver disorder * * " 16.0 

Other and unspecified digestive disease * " 13.6 

Renal disorders 

Renal failure • • " 14.4 

Other and unspecified disorders of kidney and ureter 2.4 
Mental and nervous system disorders 

Mental disorder 2.4 
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Table 6 {Continued) 



2007 WHO VA standards 2004 GBD InterVA PHMRCVA instrument % Reported in various studies 



Disease of nervous system • 3.2 

Epilepsy/Acute seizures " 4.8 

Pregnancy-, chiidbirth and puerperium-reiated disorders 

Ectopic pregnancy • 1 .6 

Abortion-related death • 4.0 

Pregnancy-induced hypertension • 9.6 

Obstetric haemorrhage • 12.8 

Obstructed iabour • 5.6 

Pregnancy-related sepsis • 8.8 

Anaemia of pregnancy • 1 .6 

Ruptured uterus • 3.2 

Other and unspecified maternal cause • * 20.8 

Perinatal causes of death 

Prematurity • * • 35.2 

Perinatal asphyxia • * * 29.6 

Neonatal pneumonia • • 8.8 

Neonatal sepsis ' " 20.8 

Neonatal tetanus • 1 0.4 

Congenital malformation • • " 27.2 

Other diseases related to the perinatal period, unspecified 12.0 

Stillbirth • 8.0 

External causes 

Road traffic accident ... g g 

Other transport accident • • 7.2 

Accidental fall • • 6.4 

Accidental drowning and submersion • * 9.6 

Accidental exposure to smol<e, fire and flames ... 7 2 

Contact with venomous animals and plants • * 3.2 

Accidental poisoning and exposure to noxious substance * * 5.6 

Intentional self-harm • • " 15.2 

Assault, homicide, war • • " 14.3 

Exposure to force of nature 0 

Lack of food and/or water 0 

Legal intervention 0 

Accident, unspecified • 14.4 

Other and unspecified external cause • • 25.6 



including PCVA, InterVA, and Population Health Metrics 
Research Consortium (PHMRC) methods, enabled the 
identification of a core group of CoD that can be certified 
by VA. This core group of CoD was mapped against the 
31 causes reported in the 2004 Global Burden of Disease 
(GBD) study to ascertain the public health importance of 
individual causes. Finally, consensus on the simplified list 
of CoD was reached in the meeting with VA experts, based 
on their experience and available evidence. 

In the 2007 WHO VA standards, there were 106 pos- 
sible CoD to be assigned by physicians, while Inter VA-3 
and InterVA-M assigned 48 causes and the PHMRC 
VA instrument reached 51 (5, 31, 39). To facilitate com- 



parison, some CoD from the WHO VA standards were 
re-categorized, creating a set of mutually exclusive, collec- 
tively exhaustive CoD categories. Table 6 displays the 
results from the review and correlation of CoD between 
the VA instruments and the GBD. 

In the review of 125 studies covering 199,158 deaths 
described above, we collated evidence on CoD certified 
by VA and reported in studies to illustrate the range 
of CoD that were observed and certifiable by VA. The top 
10 CoD reported were: 'other and unspecified cardiac dis- 
ease' (44%); 'intestinal infectious diseases' (40.8%); 'acute 
respiratory infections, including pneumonia' (37.6%); 'HIV/ 
AIDS-related death' (36.8%); 'pulmonary tuberctilosis' 
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Table 7. CoD removed from CoD list of the 2007 WHO VA 
standard and subsumed into other categories in 2012 WHO 
standard (n =27) 



Subsumed into 2012 



2007 WHO VA standard cause 


WHO VA cause 


Ottier digestive disease 


VAs-98 


Typtioid and paratyphoid 


VAs-01 .99 


Viral tiepatitis 


VAs-01 .99 


Leishmaniasis 


VAs-01 .99 


IVIalignant melanoma of skin 


VAs-02.99 


Malignant neoplasm of lymphoid, 


VAs-02.99 


haematopoietic and related tissue 




Other specified neoplasms 


VAs-02.99 


Other specified endocrine disorders 


VAs-98 


Endocrine disorders, unspecified 


VAs-98 


Other specified diseases of the respiratory 


VAs-98 


system 




Respiratory disorder, unspecified 


VAs-98 


Respiratory failure, not elsewhere 


VAs-98 


classified 




Other diseases of intestine 


VAs-98 


Disease of intestine, unspecified 


VAs-98 


Specified mental disorders 


VAs-98 


Mental disorders, unspecified 


VAs-98 


Other specified disorders of the 


VAs-98 


nervous system 




Nervous system disorders, not otherwise 


VAs-98 


classified 




Alzheimer's disease 


VAs-98 


Other specified direct maternal causes 


VAs-09.99 


Congenital viral diseases 


VAs-01 .99 


Congenital malformations of the nervous 


VAs-10.06 


system 




Other specified disorders related to 


VAs-10.99 


perinatal period 




Lack of food and/or water 


VAs-12.99 


Legal intervention 


VAs-12.99 


Accident, unspecified 


VAs-12.99 


Other specified event, undetermined intent 


VAs-12.99 



(35.2%); 'prematurity' (35.2%); 'malaria' (33.6%); 'peri- 
natal asphyxia' (29.6%); 'congenital malformations' 
(27.2%); and 'Other and unspecified external cause of 
death' (25.6%). In contrast, the 10 CoD certified and 
reported least frequently were: 'typhoid and paratyphoid' 
(0.8%); 'neoplasm of lymphoid, haematopoietic and 
related tissue' (0.8%); 'sickle cell' (0.8%); 'ectopic preg- 
nancy' (1.6%); 'anaemia of pregnancy' (1.6%); 'other and 
unspecified nutritional and endocrine disorders' (1.6%o); 
'other and unspecified disorders of kidney and ureter' 
(2.4%); 'mental disorder' (2.4%); 'pertussis' (2.4%)); and 
'disease of nervous system' (3.2%). The CoD 'Leishma- 
niasis', 'melanoma of skin', 'exposure to force of nature'. 



'lack of food and/or water', and 'legal intervention' 
have not been certified by VA in any of the reviewed 
studies. 

Elimination of CoD was based on low frequency 
reported by VA studies, not being included in the other 
VA instruments, and on experts' judgment about their 
importance, feasibility and intervention potential. As a 
resuh, 27 CoD from the 2007 WHO VA standards were 
subsumed into residual categories, including 'typhoid and 
paratyphoid', 'leishmaniasis', 'melanoma of skin', 'lack 
of food and/or water' and 'legal intervention' (Table 7). 

The inclusion of the majority of CoD in the simplified 
CoD list was based on the consistency between CoD 
from WHO VA standards against InterVA and PHMRC 
VA, GBD estimates and coverage in VA studies. All 
causes included in the GBD and the top 10 most certified 
CoD reported were retained. During expert meetings, the 
CoD 'other and unspecified non-communicable disease', 
'sepsis', 'anaemia of pregnancy' and 'ruptured uterus' 
were added to the list. Although not in the GBD or most 
commonly certified CoD, they were considered feasible 
for VA certification, provide key information to CRVS, 
contribute significant mortality burdens and are respon- 
sive to interventions. Further modifications included 
grouping related CoD not having readily distinguishable 
symptoms into broader categories. For example, 'malig- 
nant neoplasm of cervix' and 'malignant neoplasm of 
uterus' were combined into 'female reproductive neo- 
plasms'. Overall, the simplification process led to a 41.5% 
reduction in CoD compared with the WHO VA standards 
CoD hst, resulting in 60 CoD. A further two categories 
were added for fresh and macerated stillbirths, despite 
not strictly considered as CoD, because of their impor- 
tance in some settings. Table 8 presents the simplified VA 
CoD list, structuring the causes into groupings consistent 
with ICD-10 and showing in the last column how all 
ICD-10 codes map onto the 62 CoD. 

VA questionnaires and indicators 

VA questionnaires ask specific questions about signs, 
symptoms, complaints, or contextual factors that will lead 
to determining the most probable CoD. Such information 
that indicates the possibility of specific causes is inclu- 
sively termed as 'indicators'. The review aimed to collate 
evidence from field experience on: (i) specific modifica- 
tions made to VA questionnaires and their rationales; (ii) 
utility of specific indicators for CoD ascertainment; and 
(iii) identification of most and least specific indicators 
for reaching diagnoses. From the 125 studies reviewed, 
contact was attempted with 45 randomly selected authors 
(one per study, unless referred to another), and established 
with 27. Limited feedback was gathered on specific indi- 
cators, as most researchers were not able to report on 
specific modifications made to the VA instruments, and 
they found it challenging to give feedback on the utility, 
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Table 8. Simplified CoD list for 2012 WHO VA with corresponding ICD-10 codes 



2012 verbal 
autopsy code 

VAs-01 
VAs-01.01 
VAs-01 .02 

VAs-01 .03 
VAs-01 .04 
VAs-01 .05 
VAs-01 .06 
VAs-01 .07 
VAs-01 .08 
VAs-01 .09 
VAs-01. 10 
VAs-01. 11 
VAs-01 .99 



VAs-98 



VAs-02 

VAs-02.01 

VAs-02.02 

VAs-02.03 

VAs-02.04 

VAs-02.05 

VAs-02.06 

VAs-02.99 

VAs-03 

VAs-03.01 

VAs-03.02 

VAs-03.03 

VAs-04 

VAs-04.01 

VAs-04.02 

VAs-04.03 

VAs-04.99 

VAs-05 
VAs-05.01 

VAs-05.02 

VAs-06 

VAs-06.01 

VAs-06.02 

VAs-07 
VAs-07.01 

VAs-08 
VAs-08.01 



Verbal autopsy title 
Sepsis 

Acute respiratory infection, 
including pneumonia 
HIV/AIDS related death 
Diarrheal diseases 
Malaria 
Measles 

Meningitis and encephalitis 
Tetanus 

Pulmonary tuberculosis 
Pertussis 

Haemorrhagic fever 
Other and unspecified 
infectious disease 

Other and unspecified 
non-communicable disease 



ICD-10 code 
(to ICD) 



ICD-1 0 codes (from ICD) 



Infectious and parasitic diseases 
A41 A40-A41 
J22;J18 J00-J22 



B24 
A09 
B54 
BOS 

G03; G04 

A35 (obstetric A34) 

A16 

A37 

A99 

B99 



B20-B24 
A00-A09 
B50-B54 
605 

A39; G00-G05 
A33-A35 
A15-A16 
A37 

A90-A99 

A20-A38; A42-A89; B00-B19; B25-B49; B55-B99 



Oral neoplasms 
Digestive neoplasms 
Respiratory neoplasms 
Breast neoplasms 
Female reproductive neoplasms 
Male reproductive neoplasms 
Other and unspecified 
neoplasms 

Severe anaemia 
Severe malnutrition 
Diabetes mellitus 

Acute cardiac disease 
Stroke 

Sickle cell with crisis 

Other and unspecified cardiac 

disease 

Chronic obstructive 
pulmonary disease (COPD) 
Asthma 

Acute abdomen 
Liver cirrhosis 

Renal failure 

Epilepsy 



Non-communicable diseases 
R99 D55-D89; E00-E07; E15-E35; E50-E90; F00-F99; G10-G37; 

G50-G99; H00-H95; J30-J39; J47-J99; K00-K31 ; K40-K93; 
L00-L99; M00-M99; N00-N16; N20-N99; R00-R69 
Neoplasms 
C06 C00-C06 
C26 C15-C26 
C39 C30-C39 
C50 050 
057 051-058 
063 060-063 
080 007-014; 040-049; O60-D48 

Nutritional and endocrine disorders 
D64 D50-D64 
E46 E40-E46 
El 4 E10-E14 



Diseases of the circulatory system 
124 (acute ischemic) 120-125 



164 

D57 

199 



J44 



J45 (J46) 

RIO 
K74 

N19 



160-169 
D57 

110-115; 126-152; 170-199 

Respiratory disorders 
J40-J44 

J45-J46 

Gastrointestinal disorders 
RIO 

K70-K76 

Renal disorders 
N17-N19 



Mental and nervous system disorders 
G40 G40-G41 



14 

(page number not for citation purpose) 



Citation: Glob Health Action 2013, 6: 21518 - http://dx.doi.org/10.3402/gha.v6i0.2l518 



Revision of WHO verbal autopsy instrument 



Table 8 {Continued) 



2012 verbai 




iCD-10 code 




autopsy code 


Verbal autopsy titie 


(to ICD) 


ICD-1 0 codes (from ICD) 


VAs-09 


Pregnancy-, childbirth and puerperlum-related disorders 


VAs-09.01 


Ectopic pregnancy 


OOO 


OOO 


VAs-09.02 


Abortion-related death 


O06 


003-008 


VAs-09.03 


Pregnancy-induced 


013 (or 015 for 


010-016 




hypertension 


eclampsia) 




VAs-09.04 


Obstetric haemorrhage 


046 (antepartum) 


046; 067; 072 






072 (postpartum) 




VAs-09.05 


Obstructed labour 


066 


063-066 


VAs-09.06 


Pregnancy-related sepsis 


075.3 (antepartum) 


085; 075.3 






085 (postpartum) 




VAs-09.07 


Anaemia of pregnancy 


099 


O99.0 


VAs-09.08 


Ruptured uterus 


071 


071 


VAs-09.99 


Other and unspecified maternal 


O05 


001-002; 020-045; 047-062; 068-070; 073-084; 




cause 




086-099 


VAs-10 


Neonatal causes of death 






VAs-10.01 


Prematurity 


P07 


P05-P07 


VAs-10.02 


Birth asphyxia 


P21 


P20-P22 


VAs-10.03 


Neonatal pneumonia 


P23 


P23-P25 


VAs-10.04 


Neonatal sepsis 


P63 


P36 


VAs-10.05 


Neonatal tetanus 


A33 


A33 


VAs-10.06 


Congenital malformation 


089 


Q00-Q99 


VAs-10.99 


Other and unspecified perinatal 


P96 


P0O-P04; P08-P15; P26-P35; P37-P94; P96 




cause of death 






VAs-1 1 






Stillbirths 


VAs-11.01 


Fresh stillbirth 


P95 


P95 


VAs-1 1 .02 


iVIacerated stillbirth 


P95 


P95 


VAs-12 


External causes of death 






VAs-1 2.01 


Road traffic accident 


V89 


V01-V89 


VAs-1 2.02 


Other transport accident 


V99 


V90-V99 


VAs-1 2.03 


Accidental fall 


W19 


W00-W19 


VAs-1 2.04 


Accidental drowning and 


W74 


W65-W74 




submersion 






VAs-1 2.05 


Accidental exposure to smoke, 


X09 


X0O-X19 




fire and flames 






VAs-1 2.06 


Contact with venomous animals 


X29 


X20-X29 




and plants 






VAs-1 2.07 


Accidental poisoning and 


X49 


X40-X49 




exposure to noxious substance 






VAs-1 2.08 


Intentional self-harm 


X84 


X60-X84 


VAs-1 2.09 


Assault 


Y09 


X85-Y09 


VAs-1 2.10 


Exposure to force of nature 


X39 


X30-X39 


VAs-1 2.99 


Other and unspecified external 


X59 


S0O-T99; W20-W64; W75-W99; X50-X59; Y10-Y98 




cause of death 






VA-99 


Cause of death uni<nown 


R99 


R99 



value and specificity of individual questionnaire indica- (1) Structural rearrangement of order and categoriza- 
tors. The following alterations to standard instruments tion of questionnaire modules and changes in 

were reported: targeted age groups; 
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(2) Attempts to shorten the VA questionnaires by 
removal and modification of questions related to 
the duration of signs and symptoms; and 

(3) Addition of disease-specific questions for local 
conditions and research needs. 

Overall, users considered the 2007 WHO VA standards 
too long and time-consuming, expressing a desire for 
shorter and more practical instruments. This process 
of simplification was started by drafting diagnostic 
criteria for each CoD by listing symptoms indicated in 
the Oxford Text Book of Medicine (40). Subsequently, 
experts identified essential indicators for differentiat- 
ing CoD, and inclusion/exclusion was based on likely 
recognition, recollection, and reporting in VA interviews. 
Evidence of indicators' utility was gathered by correlating 
indicators from WHO VA standards, InterVA, and 
PHMRC VA procedures. Furthermore, the simplification 
of the WHO VA standards indicators was informed by 
a progressive item reduction process based on the Tariff 
method (27). Participating experts from PHMRC had 
applied the Tariff method to the PHMRC validation 
dataset and tested the effect of dropping items or sets 
of items on chance-corrected concordance and CSMF 
accuracy. These findings comprised one element of the 
discussion on the evidence base for some CoD. Indicators 
removed had low specificity and possibly generated 
answers with low reliability due to recall difficulties. 
These were mainly sub-indicators on the duration, 
frequency, and development of signs and symptoms. 
Other modifications made included the addition of 
indicators from InterVA and PHMRC VA instruments, 
the removal of overlapping indicators capturing very 
similar information, and the inclusion of social context 
indicators, facilitating use of the instrument in non- 
enumerated populations. Overall, 164 indicators were 
retained from the 2007 WHO VA standard, 57 new 
indicators introduced and 244/408 indicators from the 
2007 WHO VA standard excluded. Review by expert 
groups - for relevance to the list of causes, reliability, 
and feasibility - and comparison with machine assess- 
ment analysis led to a reduction of 45.8% in number 
of CoD-related indicators in relation to the WHO VA 
standards, resulting in a total of 221 indicators (of 
which various subsets apply to particular population 
sub-groups). 

Application of the 2012 WHO VA instrument to 
facilitate routine surveillance 

The need for consensus on simplified technical standards 
and guidelines for VA, together with their widespread 
endorsement and adoption, has become urgent. The 
systematic use of the 2012 WHO VA instrument will 
strengthen countries' CRVS systems. In the past decade, 
methodological developments in automated methods 



for VA assessment have created a shift away from 
Umited individual-level and clinical paradigms towards 
population-based epidemiological and public health think- 
ing. To facilitate application in routine surveillance 
systems, the new simpUfied VA instrument was specifi- 
cally developed for automated ascertainment of CoD. 
At present, the InterVA-4 model, as previously described 
(41), is the only available automated interpretation tool 
fully aligned with the 2012 WHO VA instrument. A 
simple, automatically interpreted VA process will lead 
to increased coverage of operational and representa- 
tive CRVS systems. Shorter and simpler interviews not 
needing physicians for CoD interpretation will facilitate 
collection of adequate data for CRVS systems. CCVA 
brings efficiency and consistency by providing a standar- 
dized interpretation of VA. The new 2012 WHO VA 
instrument will be piloted, modified, and integrated into 
national health information systems. 
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